Learning multi-objective robot control policies from demonstration

When demonstrating unknown robot tasks via teleoperation, a human user may leverage information, latent in their mind, that is not observable to the robot. Such information may include user preferences as to how a task should be performed, state information observable to the human but not the robot, or task structure information such as subtask objectives. Multiple, different actions may thus occur in what the robot perceives to be the same state. The observed mapping from perceived states to actions, π : ŝ → a, may then be a one-to-many multimap, instead of a one-to-one or many-to-one function. Our interactive learning from demonstration architecture [1] enables robot learning from teleoperative demonstration via direct policy approximation (regression). Functional regression algorithms, however, are not appropriate for learning multimap policies (Fig. 1). Instead, we have developed ROGER (Realtime Overlapping Gaussian Expert Regression), a multimap regression algorithm for interactive learning from demonstration (Fig 2). ROGER is based on the Infinite Mixture of Gaussian Processes model [2]. We achieve interactivity with a human user by reformulating it incrementally as a particle filter and using the Sparse Online Gaussian Process formulation [3]. Current work focuses on improving the algorithm’s sparse and realtime properties and applying it to real robot tasks. Whilst learning a multimap in this manner, the overall task is decomposed into a collection of overlapping, functional, experts, each corresponding to a subtask. Properly selecting an expert or subtask at run time is analogous to 0 0.2 0.4 0.6 0.8 1 −1 −0.5 0 0.5 1